To tackle the increasing complexity of designing digital electronic circuits, designers often express their design in a high-level hardware description language (HLHDL) such as Verilog HDL. The HLHDL description is then converted into an actual circuit through a process, well known to those of ordinary skill in the art as “synthesis,” involving translation and optimization. The detailed syntax and semantics of Verilog HDL is specified in the following publication that is herein incorporated by reference: “IEEE Standard Hardware Description Language Based on the Verilog Hardware Description Language,” IEEE Standard 1364-1995, Institute of Electrical and Electronic Engineers, October 1996.
An HLHDL description can be verified by simulating the HLHDL description itself, without translating the HLHDL to a lower-level description. This simulation is subjected to certain test data and the simulation's responses are recorded or analyzed.
Verification of the HLHDL description is important since detecting a circuit problem early prevents the expenditure of valuable designer time on achieving an efficient circuit implementation for a design which, at a higher level, will not achieve its intended purpose. Such an HLHDL design, whose correctness is to be determined, shall be referred to as the “design under test” or DUT. In addition, simulation of the DUT can be accomplished much more quickly in an HLHDL than after the DUT has been translated into a lower-level, more circuit oriented, description.
Conventionally, such a DUT would be tested by simulating it and applying a test stimulus to the simulation. The test stimulus often consists of multiple “stimulus vectors,” each stimulus vector being applied at a succeeding time increment. Each stimulus vector is typically a collection of binary bits, each of which is applied to a corresponding input of the design under test (DUT). The response of the DUT to the test stimulus is collected and analyzed. If the collected response agrees with the expected response then, to some degree of certainty, the DUT is believed by the circuit designer to be expressing the desired functionality. This type of conventional testing shall be referred to as “validation.”
However, even if a DUT produces entirely expected results, from validation, the circuit designer still does not know the extent to which the test stimulus has exercised the DUT. Developing a measure of the extent to which validation has “exercised” a design is referred to as “coverage analysis.”
In general coverage analysis consists of two steps: i) monitoring code execution during simulation, and ii) post simulation analysis to determine relevant items as “covered” or “not covered.” The monitoring step stores what actually took place during simulation. The post simulation analysis step examines the data stored by the monitoring step. Specifically, post simulation analysis compares what is required for complete coverage with what actually took place to determine coverage or non-coverage of the relevant items. Examples of what may be chosen as “relevant items” are: whether a statement has executed or not, whether a register has changed in value, whether a finite state machine has transitioned from one state to another and whether a sub-expression has executed.
The more basic form of coverage analysis is controllability-based. Controllability-based coverage analysis provides a measure of the extent to which the DUT has been executed as a result of the validation.
There are several conventional types of controllability-based coverage known as line coverage, conditional coverage (also known as subexpression coverage) and state coverage. Line coverage indicates the lines of code, of the HLHDL specification, which have been executed as a result of the validation. Note that for purposes of line coverage, a “line” means a particular statement of the specification language. Conditional coverage looks at each expression which determines the branching of a conditional statement and determines the number of combinations of inputs, out of the total number of combinations possible, that have been executed by each conditional statement. State coverage determines the number of states of a finite state machine, or FSM, that have been executed in comparison to the FSM's total number of possible states.
While controllability-based coverage is a very basic and important measure, there is still additional and important information that it does not provide to the circuit designer. If controllability-based coverage determines that execution of a line of the DUT has not occurred, then an important piece of information has been provided to the circuit designer: the designer knows that the test stimulus utilized in validation should be enhanced until that execution of the DUT has happened. If, however, controllability-based coverage determines that a particular execution of the DUT has occurred, it still provides no indication as to whether the result of that execution has propagated to an observable output. An observable output is just any point in the DUT where the circuit designer is collecting validation results that will be analyzed for correctness. If the results of the execution do not produce an observable output then, even if it were producing erroneous output, such errors would not be detected in the validation. Therefore, the ability to augment controllability-based coverage techniques with observability-based coverage analysis is highly desirable.